41 research outputs found
LyricJam Sonic: A Generative System for Real-Time Composition and Musical Improvisation
Electronic music artists and sound designers have unique workflow practices
that necessitate specialized approaches for developing music information
retrieval and creativity support tools. Furthermore, electronic music
instruments, such as modular synthesizers, have near-infinite possibilities for
sound creation and can be combined to create unique and complex audio paths.
The process of discovering interesting sounds is often serendipitous and
impossible to replicate. For this reason, many musicians in electronic genres
record audio output at all times while they work in the studio. Subsequently,
it is difficult for artists to rediscover audio segments that might be suitable
for use in their compositions from thousands of hours of recordings. In this
paper, we describe LyricJam Sonic -- a novel creative tool for musicians to
rediscover their previous recordings, re-contextualize them with other
recordings, and create original live music compositions in real-time. A
bi-modal AI-driven approach uses generated lyric lines to find matching audio
clips from the artist's past studio recordings, and uses them to generate new
lyric lines, which in turn are used to find other clips, thus creating a
continuous and evolving stream of music and lyrics. The intent is to keep the
artists in a state of creative flow conducive to music creation rather than
taking them into an analytical/critical state of deliberately searching for
past audio segments. The system can run in either a fully autonomous mode
without user input, or in a live performance mode, where the artist plays live
music, while the system "listens" and creates a continuous stream of music and
lyrics in response.Comment: 15 pages, 9 figures, 2 table